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2.  Evaluation  Division  of  the  Directorate  of  Evaluation  and  Standardization 
(DES)  was  tasked  to  assess  the  effectiveness  of  training  in  the  FWMEQC  early 
in  Septeirber  1980.  The  evaluation  plan  included  provisions  for  investigation 
of  the  adequacy  of  techniques  and  procedures  of  instruction.  It  further 
included  observation  of  a*  selected  sanple  of  academic  and  flight  training  to 
evaluate  instruction,  training  materials,  and  training  areas.  Primary 
objective  of  the  observations  was  to  determine  how  well  course  objectives  were 
being  achieved  by  students. 

3.  The  evaluation  produced  the  following  major  findings: 

a.  Instructional  systems  development  procedures  and  techniques  were 
adequate. 

b.  The  Program  of  Instruction  (POI)  has  been  updated  and  was  forwarded  to 
TRADOC  for  approval  on  30  July  1980. 

c.  There  is  a  high  degree  of  consistency  on  training  objectives  from  the 
POI  to  lesson  plans  and  student  handouts. 

d.  Training  objectives  are  being  satisfactorily  achieved. 

e.  Except  for  a  limited  number  of  test  questions,  examinations  are 
satisfactory. 

f.  Evaluation  of  training  through  analysis  of  exam  results  is  not  being 
adequately  accomplished  in  the  FWMEQC. 
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ABSTRACT 


1.  The  Fixed  Wing  Multi-Engine  Qualification  Course  (FWMEQC)  is  10 
weeks,  2  days  in  length.  Academic  instruction  is  conducted  by  the 
Department  of  Academic  Training.  Flight  instruction  is  provided  by  a 
civilian  contractor  under  the  supervision  of  the  Department  of  Flight 
Training.  The  course  was  established  to  train  commissioned  and  warrant 
officer  Army  aviators  in  fixed  wing  aircraft  flight  techniques  for 
multi-engine  aircraft  qualification  including  the  Army  Fixed  Wing  Instru 
raent  Rating. 

2.  Evaluation  Division  of  the  Directorate  of  Evaluation  and  Standardi¬ 
zation  (DES)  was  tasked  to  assess  the  effectiveness  of  training  in  the 
FWMEQC  early  in  September  1980.  The  evaluation  plan  included  provisions 
for  investigation  of  the  adequacy  of  techniques  and  procedures  of  in¬ 
struction.  It  further  included  observation  of  a  selected  sample  of 
academic  and  flight  training  to  evaluate  instruction,  training  materials 
and  training  areas.  Primary  objective  of  the  observations  was  to  deter¬ 
mine  how  well  course  objectives  were  being  achieved  by  students. 

3.  The  evaluation  produced  the  following  major  findings: 

a.  Instructional  systems  development  procedures  and  techniques  were 
adequate. 

b.  The  Program  of  Instruction  (POI)  has  been  updated  and  was  for¬ 
warded  to  TRADOC  for  approval  on  30  July  1980. 

c.  There  is  a  high  degree  of  consistency  on  training  objectives 
from  the  POI  to  lesson  plans  and  student  handouts. 

d.  Training  objectives  are  being  satisfactorily  achieved. 

e.  Except  for  a  limited  number  of  test  questions,  examinations  are 
satisfactory. 

f.  Evaluation  of  training  through  analysis  of  exam  results  is  not 
being  adequately  accomplished  in  the  FWMEQC. 
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EVALUATION  OF  THE  FIXED  WING 
MULTI-ENGINE  QUALIFICATION  COURSE 

1.  INTRODUCTION: 

a.  Background.  The  Fixed  Wing  Multi-Engine  Qualification  Course 
(FWMEQC)  was  established  to  train  commissioned  and  warrant  officer  Army 
aviators  in  fixed  wing  aircraft  flight  techniques  for  multi-engine 
aircraft  qualitication.  Graduates  receive,  the  award  of  an  Army  Fixed 
Wing  Instrument  Rating  in  accordance  with  FAA  standards  and  applicable 
Army  regulations.  Length  of  the  course  in  peacetime  is  10  weeks,  2  days 
with  a  total  of  416  hours  of  instruction.  Academic  instruction  is 
conducted  by  the  Department  of  Academic  Training  and  flight  instruction 
is  provided  by  Doss  Aeronautical  Services,  Inc.,  under  the  supervision 
of  the  Department  of  Flight  Training. 

b.  Purpose.  Since  this  program  of  instruction  has  been  in  operation 
for  a  number  of  years  and  has  not  been  formally  evaluated,  this  assess¬ 
ment  of  its  overall  effectiveness  was  initiated. 

2.  EVALUATION  PLAN: 

a.  Objectives. 

(1)  Determine  whether  techniques  and  procedures  applied  during 
the  instructional  systems  development  of  the  FWMFQC  were  adequate. 

(2)  Determine  by  spot-checks  of  selected  samples  of  academic 
and  flight  training  if  training  materials,  instruction,  and  evaluation 
are  effective  in  accomplishing  training  objectives. 

(3)  Determine  whether  there  are  any  significant  problems 
existing  or  developing  during  the  training  process. 

b.  Methodology. 

(1)  Research  the  instructional  systems  development  process  by 
interviews  with  personnel  who  are  proponents  for  each  phase  of  develop¬ 
ment  to  determine  adequacy  of  course  development. 

(2)  Compare  training  objectives  in  the  program  of  instruction 
(POI),  the  lesson  plans,  and  student  handouts  for  consistency. 

(3)  Observe  a  selected  sample  of  academic  instruction  to 
determine  whether  training  objectives  were  achieved  and  whether  training 
facilities  are  adequate.  Include  at  least  one  examination  in  these 
observations. 


(4)  Visit  che  flight  line  and  observe  training  to  determine 
adequacy  of  training  activities  and  whether  ary  training  problems  exist. 

(5)  Apply  the  following  techniques  to  determine  any  significant 
trcnns  or  problems: 

(a)  Review  student  critiques. 

(b)  interview  flight  instructors,  academic  instructors, 
and  students.  Research  into  the  techniques  and  procedures  applied  by 

1TTD  during  instructional  systems  development  shows  that  they  were  adequate. 

(c)  Interview  standardization  instructor  pilots  in  Flight 
Standardization  Division  (FSD),  DES. 

c.  Conduct  of  the  Evaluation.  The  above  methodology  was  executed 
i»y  the  two  officers  named  in  page  ii  under  che  supervision  of  the 
Commander  of  Internal  Instructional  Systems  Evaluation  Branch,  Evaluation 
Division,  DES. 

3.  FINDINGS : 

a.  Research  in  the  Directorate  of  Training  Development  into  the 
techniques  and  procedures  applied  during  instructional  systems  development 
show  that  they  were  adequate.  An  analysis  of  the  existing  course,  which 
was  developed  prior  to  the  School  Model  76  requirements,  led  to  the 
conclusion  that  the  course  meets  the  training  needs  of  the  U.  S.  Army 
Aviation  Center.  Toe  research  also  revealed  that  the  several  subcourses 
in  FWMEQC,  i.e..  Weather  Flight  Planning,  IFR  Flight  Planning,  Aircraft 
"•••<! ftn  i.  etc.,  have  been  kept  up-to-date  as  needed.  Additionally, 

••csearch  proved  that  the  PCI  has  been  recently  updated  and  was  presented 
to  iRADOC  for  approval  on  30  July  1980.  That  P01  was  a  major  change 
including  new  lesson  plans,  examinations,  and  some  subject  matter  revisions. 
TRABOC  approved  the  new  POI  on  13  November  1980. 

b.  Comparison  of  training  objectives  listed  in  the  POI  with  those 
!  'error  plans  and  student  handouts  shows  very  good  consistency. 

be  «cn  -t  lar.s  expand  POT  learning  objectives  .ito  their  realistic  inherent 
leeti.ii.g  elements. 

c.  Observations  of  sixteen  hours  of  academic  instruction  produced 
the  following  information  which  shows  that  training  materials  and 
instruction  are  effective  in  accomplishing  training  objectives. 

(1)  Classrooms,  the  learning  resource  center,  and  training  areas 
are  adequate  to  support  the  training  mission. 


(2)  Training  aids  are  operable  and  well  maintained. 


(3)  Instructor  training  is  adequate  and  instructor  motivation 
is  uniformly  high. 

(4)  Student  response  during  the  observations  and  the  demonstra¬ 
tion  of  subject  matter  expertise  by  instructors  indicate  learning  objec¬ 
tives  are  being  achieved. 

(5)  Observation  of  two  examinations,  68-EA2E,  Principles  of 
Flight,  and  68-EA3C,  Weather  Flight  Planning,  led  tc  the  following. 

(a)  Overall  results  of  the  two  exams  for  nine  classes 
indicate  students  are  succeeding  in  the  two  subject  STeas.  The  following 
table  illustrates  that  success. 


Test  Number 

68-EA-2E 

68-EA-3C 

Number  of  Questions 

33 

40 

Number  of  Scores  Checked 

107 

1C7 

Standard  Deviation 

6.4 

7.1 

Average  Score 

91 

89 

Time  Allowed 

1  hour 

1  hour 

(b)  Further  investigation  of  the  effectiveness  of  the  two 
exams  through  item  analysis  was  undertaken.  Only  one  item  analysis  for 
68-EA-3C  could  be  found.  That  one  was  dated  January  1980  and  included 
only  ten  students.  Answer  sheets  for  both  exams  were  available  In  the 
Department  of  Academic  Training  for  nine  classes  (80-1  through  80-9,  107 
students).  They  were  obtained  by  the  evaluators  and  analysis  was  conducted. 
Results  are  shown  at  Annexes  A  &  3.  Review  of  these  results  shows  that: 

On  68-EA2E,  only  one  question,  number  26,  was  missed 
by  30%  or  more  of  students. 

2  On  68-EA-3C,  there  are  three  questions  where  the 
percent  of  misses  exceeds  30%  (numbers  17,  21  and  29).  NOTE:  Selection 
of  30%  misses  on  a  test  question  as  a  critical  point  is  based  upon  the 
traditional  use  of  70%  correct  as  the  passing  point. 

d.  Observation  of  Training  at  the  flight  line  produced  these  results. 

(1)  Eighty-three  EFF1  (contact  flight  grade)  and  EFF2  (Instru¬ 
ment  flight  grade)  check  rides  were  reviewed.  There  were  only  two 
unsatisfactory  grades  given.  The  two  students  involved  passed  their 
rechecks.  Therefore,  flight  instruction  and  flight  training  facilities 
are  judged  as  adequate.  A  breakdown  of  the  range  of  grades  for  check 
rides  for  eighty-three  students  on  EFF1  and  EFF2  is  shown  at  Annex  C. 
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(2)  For  each  check  ride  CEFF1  and  EFr2),  the  number  of  downgrades 
(C  and  U)  on  each  flight  cask  were  counted.  The  results  are  shown  at 
Annex  C.  The  largest  number  of  down^raaes  given  for  any  flight  task  was 
six  for  FFF1  and  five  ior  EFF.’ .  This  is  further  indication  that  flight 
training  does  include  detailed  evaluation  of  students  on  all  flight 
tasks  and  that  flight  training  is  adequate. 

(3)  Observations  of  flight  simulator  training  shows  it  to  be 
outstanding.  Review  of  critique  sheets  by  students  for  simulator  training 
showed  ihev  felt  instructors  were  superior.  The  only  adverse  comment  on 
this  training  concerned  the  availability  and  maintenance  of  the  GAT-2, 
investigation  of  this  comment  with  the  branch  chief  and  with  maintenance 
personnel  showed  that  down  time  for  the  trainers  has  presented  no  handicap 
to  training.  There  are  two  trainers  available  and  maintenance  records 
show  that  no  overtime  pay  has  been  necessary  for  keeping  at  least  one  of 
them  operable  at  all  times. 

(4)  Performance  data  on  each  student  and  flight  instructor  is 
being  maintained  by  the  Department  of  Flight  Training.  However,  no 
formal  reports  are  being  made  to  document  the  performance  of  each  class 
nor  to  show  a  profile  of  instructor  strengths  or  weaknesses.  The  reason 
given  for  not  making  such  reports  was  that  student  load  does  not  justify 
the  expense  required. 

e.  Interviews  with  flight  and  academic  instructors  and  review  of 
student  critiques  did  not  indicate  the  existence  of  any  significant 
training  problems  or  any  trends  toward  problems.  There  were  some  comments 
which  show  how  instructors  perceive  factors  that  do  have  some  impact  on 
training.  They  included  the  following: 

(1)  light  instructors  were  consistent  in  the  contention  that 
students  are  not  fully  versed  in  the  basic  fundamentals  of  fixed  wing 
flight  and  that  the  training  program  is  not  adequately  preparing  the 
student:  for  fixed  wing  aviator  assignments.  NOTE:  This  comment  is  not 
.upper ted  by  the  grade*-,  gl\»n  by  the  flight  instructors. 

(2'  lustre'' tors  are  concerned  with  the  age  and  condition  of  the 
T-42  aircraft  While  availability  is  very  high,  near  B07 ,  these  aircraft 
have  an  vtr a,»e  o*  •2. 4a  .6  airframe  hours,  all  in  a  training  environment. 
A  breakdown  of  aircraft  airframe  hour^  is  shown  at  Annex  E. 

4.  CONCLUSIONS; 

a.  Procedures  applLed  durLng  the  instructional  systems  development 
of  the  FVIMEQC  were  adequate. 


4 


b.  The  POI  has  been  updated  and  was  submitted  to  TRADOC  for  approval 
on  30  July  1980.  It  was  approved  on  13  November  1980. 

c.  There  is  a  high  degree  of  consistency  on  training  objectives 
from  the  POI  to  lesson  plans  and  student  handouts. 

d.  Training  objectives  in  the  classroom  and  on  the  flight  line  are 
being  achieved  satisfactorily. 

e.  Except  for  a  limited  number  of  test  questions,  examinations  are 
satisfactory. 

f.  Evaluation  of  training  through  analysis  of.  exam  results,  as 
required  by  USAAVNC  Reg  350-14,  is  not  being  adequately  accomplished  in 
the  FWMEQC. 
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ANNEX  A 


EVALUATION  CF  GRADE  RANGE  EXAM  68-EA-2A  PRINCIPLE  OF  FLIGHT 


SCORE 

80-1 

80-2 

80-3 

80-4 

80-5 

I  80-6 

80-7 

I  80-8 

80-9 

TOTAL 

100 

2 

2 

2 

2 

3 

fl 

1 

■ 

13 

97 

2 

1 

1 

1 

fl 

1 

fl 

5 

15 

94 

3 

2 

4 

1 

3 

fl 

3 

fl 

5 

27 

91 

1 

1 

3 

1 

2 

B 

2 

2 

16 

88 

2 

1 

2 

2 

2 

2 

11 

85 

2 

2 

2 

6 

82 

1 

2 

4 

1 

2 

2 

12 

79 

1 

1 

2 

1 

5 

76 

1 

1 

73 

1 

1 

NUMBER 

OF  SC 

ORES  R 

EVlEWfl 

D: 

12 

11 

12 

13 

11 

12 

12 

12 

12 

107 

AVERAG 

E: 

92 

90 

93 

88 

93 

91 

89 

_ 

91 

95 

91 

TOTAL 


TOTAL  X 


3 

28 

12 

23 

25 

36 

6 

2 

12 

15 

7 
3 

19 

3 

8 
2 

15 


3 

26 

11 

21 

23 

34 

6 

2 

11 

14 

7 

3 

18 

3 

7 
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ANNEX  C 


FLIGHT  EVALUATION  GRADE  RANGE  FOR  CLASSES  80-1  THROUGH  80-7 


EFF  1  EFF  2 

GRADE  #  Z  #  X 


95 .  1  0.8  9  7.4 

94 .  3  2.4  0 

93 .  2  1.6  4  3.3 

92 .  12  9.9  12  9.9 

91 . 6  4.9  1  0.8 

90 .  11  9.1  9  7.4 

89 .  4  3.3  6  4.9 

88 .  13  10.7  12  9.9 

87 .  13  10.7  5  4.1 

86 .  2  1.6  6  4.9 

85 .  3  2.4  13  10.7 

84 .  2  1.6  1  0.8 

83 .  3  2.4  0 

82 .  1  0.8  2  1.6 

81 .  2  1.6  0 

80 .  4  3.3  0 

79 .  1  0.8  1  0.8 

78 .  0  —  1  0.8 

72 .  0  —  1  0.8 

AVERAGE:  88.2  87.7 


ANNEX  D 


SUMMARY  OF  DOWNGRADES  FOR  FWMEQC  CHECKRIDES 


MANEUVERS 


EFF  1 

I  OF  STUDENTS 
DOWNGRADES 


EFF  2 

#  OF  STUDENTS 
DOWNGRADES 


ANNEX  D 


MANEUVERS 

EFF  1 

#  OF  STUDENTS 
DOWNGRADES 

EFF  2 

#  OF  STUDENTS 
DOWNGRADES 

Orientation 

2 

Oral  Examination 

1 

Track  Interception 

3 

Track  Following 

4 

Radio  Fix  Iden 

2 

Transition 

1  • 

Holding 

3 

Approach  (GCA) 

1 

Approach  (VOR) 

5 

Approach  (NDB) 

3 

Approach  (ILS) 

4 

Visual  Approach 

2 

Missed  Approach 

TOTAL  DOWNGRADES: 

1 

4* 

45 

D-2 


ANNEX  E 

T-42  AIRCRAFT  AIRFRAME  HOURS  AS  OF  12  SEP  80 

YEAR  SERIAL  it  AIRFRAME  H0UR£?2  •  ^L» 


»  *U  1 

1  ♦ 


65 

12682 

11827.1 

65 

12687 

13083.0 

65 

12688 

12138.5 

65 

12700 

11490.8 

65 

12724 

13273.5 

66 

04300 

13135.3 

66 

04301 

12998.7 

66 

04305 

12825.6 

66 

04308 

12577.1 

66 

04309 

11067.1 

TOTAL: 

124416.7 

AVERAGE: 

12441.6 

8  * 

<>7  uiv«'> 


NOTE:  Data  provided  by  Mr.  Smith,  Northrop  Aviation. 
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ATTN:  ATTNG-EV 

Fort  Monroe,  VA  23651  2 

Commandant 

US  Army  Air  Defense  School 
ATTN:  Director  of  Evaluation 
Fort  Bliss,  TX  79916  1 

Commandant 

US  Army  Armor  School 

ATTN:  Director  of  Evaluation 

Fort  Knox,  KY  40121  1 

Commandant 

US  Army  Engineer  School 
ATTK;  Director  of  Evaluation 
Fort  Tielvoir,  VA  22060  1 


Commandant 

US  Army  Infantry  School 
\TTN:  Director  of  Evaluation 
fort  Benning,  GA  31905  1 


Commandant 

US  Army  Transportation  School 
ATTN:  Director  of  Evaluation 
Fort  Eustis,  VA  23604  1 

Commandant 

US  Army  Field  Artillery  School 
ATTN:  Director  of  Evaluation 
Fcrt  Sill,  OK  73503  1 

Commandant 

TJS  Army  Intelligence  School 
ATTN:  Director  of  Evaluation 
Fort  rtuachuca,  AZ  85613  1 


Commandant 

US  Army  Signal  School 

ATTN:  Director  of  Evaluation 

Fort  Gordon,  GA  30905  1 


Commandant 

US  Army  Institute  of  Administration 
ATTN:  Director  of  Evaluation 
Fort  Benjamin  Harrison,  IN  46216  1 

Commandant 

US  Army  Military  Police  School 
ATTN:  Director  of  Evaluation 
Fort  McClellan,  AL  36201  1 

Commandant 

US  Army  Chaplain  School 

ATTN:  Director  of  Evaluation 

Fort  Wadsworth,  NY  10305  1 

Commandant 

US  Army  Communciations-Electronics 
School 

ATTN:  Director  of  Evaluation 
Fort  Monmouth,  NJ  07703  1 

Commandant 

US  Army  Institute  for  Military 
Assistance 

ATTN:  Director  of  Evaluation 
Fort  Bragg,  NC  28307  1 

Commandant 

US  Army  Missile  &  Munitions  School 
ATTN:  Director  of  Evaluation 
Redstone  Arsenal,  AL  35809  1 

Commandant 

US  Army  Ordnance  School 
ATTN:  Director  of  Evaluation 
Aberdeen  Proving  Ground,  MD  21005  1 

Commandant 

US  Army  Quartermaster  School 
ATTN:  Director  of  Evaluation 
Fort  Lee,  VA  23801  1 


Commandant 

US  Army  Sergeants  Major  Academy 
ATTN:  Director  of  Evaluation 
Fort  Bliss,  TX  79918  1 

Commandant 

US  Army  Element,  School  of  Music 
ATTN:  Director  of  Evaluation 
Norfolk,  VA  23521  1 

Commandant 

Defense  Information  School 

ATTN:  Director  of  Evaluation 

Fort  Benjamin  Harrison,  IN  46216  1 

Commandant 

Defense  Language  Institute 
ATTN:  Director  of  Evaluation 
Presidio  of  Monterey,  CA  93940  1 

Commandant 
Defense  Language 
English  Language  Center 
ATTN:  Director  of  Evaluation 
Lackland  AFB,  TX  78236  1 


Commander 

US  Army  Aviation  Center 

ATTN:  ATZQ-DCG 

Fort  Rucker,  AL  36362 

Commander 

US  Army  Aviation  Center 

ATTN:  ATZQ-ES 

Fort  Rucker,  AL  36362 

Commander 

US  Army  Aviation  Center 

ATTN:  ATZQ-T 

Fort  Rucker,  AL  36362 

Commander 

US  Army  Aviation  Center 

ATTN:  ATZQ-TD 

Tort  Rucker,  AL  36362 


